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Abstract — In this paper, we investigate the optimal tradeoff be- 
tween source and channel coding for channels with bit or packet 
erasure. Upper and Lower bounds on the optimal channel coding 
rate are computed to achieve minimal end-to-end distortion. The 
bounds are calculated based on a combination of sphere packing, 
straight line and expurgated error exponents and also high rate 
vector quantization theory. By modeling a packet erasure channel 
in terms of an equivalent bit erasure channel, we obtain bounds 
on the packet size for a specified limit on the distortion. 

Index terms-Joint source and channel coding, binary erasure 
channel, packet erasure, error exponent, high rate vector quan- 
tization. 

I. Introduction 

In [1] Shannon presented his celebrated result on the 
asymptotic optimality of separable source and channel coding. 
However, for finite block length systems, the importance and 
superior performance of joint source channel coders has been 
well recognized and is an area of active research (see for exam- 
ple [9]). Specific research thrusts have included investigating 
source coders that incorporate channel information in the 
design, channel coders that provide unequal error protection 
to various source bits, and iterative source-channel decoders. 

An important issue in joint source channel coding is the 
tradeoff between source and channel coding rates. For a 
fixed source vector dimension and channel capacity, there 
is a tradeoff between the source and channel coding rates. 
A high rate channel code implies more bits for the source 
coder which results in a high quality representation at the 
source but has a higher probability of being received in error. 
Similarly, a low rate channel code results in fewer bits for 
the source coder; consequently, the representation is of lower 
quality at the source but there is a higher probability of 
being received without error at the receiver. This tradeoff has 
been quantified for binary symmetric channels (BSC) [2] and 
Gaussian channels [3]. 

This paper addresses the problem of optimal allocation 
of rate between a source encoder and a channel encoder 
for transmission over erasure channels. The system under 
investigation is a concatenation of a vector quantizer with a 
channel coder and the objective is to minimize the end-to-end 
distortion. Upper and lower bounds on the channel coding 
rate are constructed that minimizes the end-to-end distortion. 
The upper bound on channel coding rate is derived using the 
sphere packing and straight line exponents as a bound for 



performance of the channel code. Similarly, the lower bound 
on rate is derived based on the expurgated error exponent for 
the erasure channel. The proposed bounds suggest that the 
optimal channel coding rate is substantially smaller than the 
channel capacity. Asymptotically, as the erasure probability 
e — > 0, the optimal channel coding rate equals 1 . The resulting 
upper and lower bounds are then adapted to obtain the optimal 
coding rate for packet erasure channels. 

The closed form approximations for the optimal coding 
rate are derived under the assumption of asymptotically small 
erasure probabilities. Also, a high rate quantization regime is 
considered and hence, distortion achieved asymptotically with 
large k equals the rate distortion bound. The proposed bounds 
are independent of the source distribution for sources with 
fixed dimensionality and finite support. 

The rest of the paper is organized as follows. In Section 2, 
we define the system and present the notation and assumptions 
made in this paper. In Section 3 we evaluate the upper 
and lower bounds on the rate for erasure channels using 
the expurgated, sphere packing and straight line bounds. In 
Section 4 we present some numerical results and conclude in 
Section 5. 

A model of the communication system under investigation 
is given in Fig. 1. Consider a random vector X G 5R fc that 
has a probability density function / over support set A, a 
closed bounded subset of IStr with nonempty interior. Let 
X be quantized by a vector quantizer Q : K fe — > C where 
C = [yii v 2i ■ -- , yj\/] is the codebook of the vector quantizer 
with m = logM bits per source symbol. All logarithms are 
to base 2. Consequently, the quantizer can be modeled as [4], 

M 

q(*)=5>* 1s *oo (d 

i=l 

where {Si}^L 1 is a partition of 5ft fc into disjoint regions, each 
of which is represented by y i and 1^ (.) is the indicator 
function which equals 1 if x lies in the i th cell of the partition. 
The average distortion using this quantizer is given by, 

M 

D m (Q) = V / \\x-yAYf{x)dx (2) 
= 2-' }Rr+0(1 \ (3) 
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Fig. 1. System Block Diagram 



where Q follows from Zador's distortion formula [5] and p is 
the power of the distortion measure. Traditional quantization 
theory has worked on computing optimal quantizers that 
achieve the infimum of D m (Q). The quantizer design has 
been expanded to include the effect of channel errors; however, 
the problem is extremely challenging and little analytical 
results are known in such cases. 

In our system, we consider that the m-bit source codewords are 
first randomly permuted using a mapping it and then passed 
to a channel encoder of rate r — m/n before transmission 
over a binary erasure channel with erasure probability e. For 
simplicity, we have included the index assignment it as part of 
the source encoder. The results are independent of the index 
assignment. The channel encoder generates a unique n-bit 
channel codeword for each of the m-bit source codewords. 
The added redundancy n — m is used to protect the source 
code word from channel impairments. The ttansmission rate 
per source component is R = n/k, and the quantization rate 
is R s = m/k — Rr. Following the notations used in [2], 
we denote a{ = O (bi) if |ai|/6i < c for some c > and i 
sufficiently large. We denote — fl(bi) if \ai\/bi > c for 
some c > and sufficiently large i. Finally, = o (bi) if 
lim^oo di/bi = 0. 

II. Binary Erasure Channel 

We now consider obtaining bounds on the coding rate for 
a binary erasure channel (BEC). The end-to-end distortion for 



the system in Fig. 1, is readily given by [2] 

M „ 

D R {Q,e)=Y,l(j\i) I Hx-yjn^)^ (4) 

i,j=l Si 

where e is the bit erasure probability and q(j\i) is the 
conditional probability that the channel decoder decides in 
favor of the j th channel codeword when the i th codeword 
was transmitted. 

A. Lower bound on channel coding rate 

The lower bound on the channel coding rate is obtained by 
upper bounding the distortion at the decoder. Assuming small 
bit erasure probability and following [2], the total distortion 
may be upper bounded as, 

D R (Q, e) < D m (Q) + 0(1) ww^ P e \ t (5) 

In (|5}, the total distortion is the sum of the distortion due to 
the vector quantizer ( D m (Q)), and the distortion due to the 
errors in transmission. The positive O(l) term is due to the 
fact that / has support A, and yj is contained in A for all 
j [2]. The problem of interest is posed as follows: "Given a 
binary erasure channel with R and source X G find the 
optimal rate r that minimizes the distortion Dr (Q, e, )" 

For an arbitrary binary discrete memoryless channel, Shan- 
non's channel coding theorem guarantees that for channel 



code rates r below capacity, the probability of error is upper 
bounded by [6] 



max P eH < 2- nE "^ +ol - r \ 

Ki<M 1 ~ 



(6) 



where, E ex (r) is the expurgated error exponent and is an 
exponentially decreasing function of the rate. The dependence 
of P e u on n indicates that the decoding error probability can be 
decreased by increasing the length of the channel codewords. 
The expurgated error exponent is given by [6], 
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where, 



E x (p, q) = 
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Note that q = [^(0) q(l) ...q(K — 1)] represents the 
probability of the input channel alphabets and P{j\i) is the 
probability of receiving output symbol j when input symbol 
i is transmitted. In (JSJ), K and J represent, respectively, the 
cardinality of the input and output alphabets of the channel. 
For a binary erasure channel, J — 3 and K = 2. For the binary 
erasure channel,the transition probability matrix is given by 
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TABLE I 

Transition Probability Matrix with elements P(j\i) for BEC. 



For a symmetric channel, the q that maximizes the error 
exponent is the uniform probability assignment [7]. Thus for 
the binary erasure channel, q = [q (0) q(l)] = [0.5 0.5]. 
Substituting the upper bound for the probability of error 
into (|5}, we obtain the end-to-end distortion as 

D R (Q, e, tt) < 2- pRr+ °^ + 2 - fc ^e*H+o(r) (9) 

Consider the case of large R: To ensure that neither of the two 
terms on the right hand side of l|9} dominates the distortion 
upper bound, we choose the exponents of the two terms to be 
within o(l) of each other [2], [8]. Hence, we set 

V 



E ex (r) 



•o(l) 



(10) 



to obtain the channel coding rate that optimizes the end-to-end 
distortion at the decoder. This optimal rate is characterized by 
Theorem 1, which is similar to Theorem 1 in [2]. 

Theorem 1: The upper bound on the minimum p th power 
distortion, averaged over all index assignments of a k- 
dimensional cascaded good vector-quantizer and channel en- 
coder that transmits over a binary erasure channel with bit 



erasure probability e, is achieved with a channel code rate r e 
satisfying 



loglog(l/e) + loge + c e 



log(l/e) 



-O 



log log (1 A) 
log 2 (1/e) 



o(l) 



where, c e satisfies 



P c e _ (P[ k ) (Igg l °g (1/e) + log e + c £ ) 
k 



log (1/e) 



Proof: 

For a BEC, evaluating (8), we obtain 



(11) 



-1 = 0. 
(12) 



max E x (p, q) 
q 



1 - log 1 



(8) and thus the expurgated error exponent becomes 



E P 



) = sup jp [l - r - log (l + e 1 /")] } 



(13) 



(14) 



The p which maximizes the error exponent and also satis- 
fies J10I is given by 

log (1/e) 



r log log (1/e) + c e (15) 

Substituting for p and c c into dlOi we obtain the optimal 
rate ( II \\ and hence the theorem is proved. □ 
Note that the expression for the expurgated error joint source 
channel rate is similar to the BSC case [2] with the difference 
being the argument of the log term. Appendix 1 in [2] provides 
details on the derivation for p and c e . A further simplification 
in the expression for the rate can be obtained by neglecting 
the 0(1) and o(r) terms and equating fl!4i to the exponent 
of the source coding distortion yielding 



P 



1 - log 1 + e 



(16) 



Numerical values of r ex is given in Figure 2 and are explained 
in Section IV. 

B. Upper bound on channel coding rate 

Following the analysis in [2], the upper bound on the 
average distortion minimized over all channel code rates for 
large R and small bit error probability for the binary erasure 
channel can be obtained as 

1 M 

D R (Q, e, 7T) > An (Q) (1 - P e ) + (1) — P e\k 



k=l 



2 -pRr+0(l) (1 _p e ) + r2(l)p e 



(17) 



where P e is the probability of error occurring in the channel. 
A lower bound on this probability of error is given by, 



p > 2-nE sl (r)+o(n) _ 2-kRE sl (r)+o(R) 



(18) 



where E s i is the straight line exponent. The straight line 
exponent E s i (r) is a linear function of r which is tangent 



to the sphere packing exponent E sp (r) and also satisfies 
E s i (0) = E ex (0). The sphere packing exponent [6] is given 
by, 



E sp (r) 



sup 

p>0 



-pr 



maxE'o (p,q) 
q 



(19) 



and, 



max (p, q) 
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The straight line exponent can be written as, 



E„i (r s i)=E ex (Q)+r sl 



[E sp (r') - E ex (0)] 



(20) 



(21) 



where, r' is the rate at which the straight line exponent meets 
the sphere packing exponent tangentially. The straight line ex- 
ponent has also been characterized in [10] for a binary erasure 
channel. Now, the end-to-end distortion is thus bounded as 



D R (Q,e,ir) > 2 ^ Rr + ^ +2- kRE ^+°^ 



(22) 



The channel coding rate that minimizes this bound is now 
characterized in Theorem 2. 

Theorem 2: An upper bound on the channel code rate r that 
minimizes the p th power distortion averaged over all random 
index assignments of a fc-dimensional cascaded good vector 
quantizer for a binary erasure channel with small effective bit 
erasure probability e and large R is given by 



r s i 



E ex (0) 



-E sp (r')--Eex(0) 



(23) 



Proof: As in the earlier case, for large R, to prevent either 
of the terms in the distortion bound i22\ from dominating 
the other, we set the straight line exponent to be linearly 
proportional to the exponent term of the noiseless-optimal 
distortion within o(l) of each other. Thus, 



E si (r) = |r si +o(l) 



(24) 



Substituting d2U in d24t> . the theorem is proved □. 

Note that to completely characterize r s i we need to explic- 
itly evaluate the sphere packing exponent E sp (r). It is easily 
seen that a uniform probability assignment for the input states 
to the channel q(.) maximizes E a (p,q) and thus E sp {r) can 
be evaluated as, 



E sp (r) = sup{/o(l 

p>0 



log[(l-e) + e2"]} 



(25) 



Note that (12511 is a concave function of p and hence the 
supremum can be replaced by the max operator. The p which 
maximizes (125 \ satisfies 



(l-e) + 2Pe 



(26) 



We can use this relation between the rate and p to express 
the sphere packing exponent in terms of the channel encoding 



rate for a given erasure channel as, 

E sp (r) = r log r + (1 — r) log(l — r) — r log 



1 - e 



-loge 
(27) 

At r', the slope of the sphere packing exponent equals the 
slope of the straight line exponent. Thus, 



dEs 



E sp (r') - E ex (0) 



(28) 



dr r' 
Differentiating (12711 and substituting in j28L we get 

r' = 1 - 2 Ec!C (°)- I °g( 1 /«) (29) 
It turns out that E sp (r') is nearly for small values of e. 

III. Numerical Results 

The bounds derived above for the erasure channel can be 
easily extended to the case of packet erasures. We use a 
simplified model for the packet erasure channel and assume 
that a packet erasure occurs if any of the bits within the 
packet suffers an erasure. Although this assumption simplifies 
the packet erasure channel model, it is useful in obtaining 
closed form bounds on the coding rate over such channels. 
For a packet of size P bits, the probability of a packet 
erasure S is given by 5 = 1 — (1 — e) P , where as before e 
denotes the probability of bit erasure. The error exponent for 
a binary erasure channel with erasure probability e and a 2 P - 
ary erasure channel with erasure probability 5 is the same. 
Hence, given the packet erasure probability S, we consider an 
equivalent binary erasure channel with bit erasure probability 
e = 1 — (1 — §) x l p and find the bounds on the rate and 
distortion for the corresponding BEC. 

The plot of the upper and lower bound on channel coding 
rate as a function of the erasure channel probability for k=4 
and squared distortion measure is given in Fig. 2. The bounds 
for various packet sizes P = 1, 10 and 100 are shown in 
Fig. 2 . It is observed that for a given packet size, as the 
erasure probability increases, the channel coding rate decreases 
indicating that more bits need to be invested on channel coding 
to combat a hostile channel. Further, for a given packet erasure 
probability, as the packet size increases, the channel coding 
rate increases implying that more bits can be allocated for 
source coding with larger packet size. Fig. 3 offers a different 
perspective on the results. From the bounds on the channel 
coding rate, we can get the bounds on the distortion due to 
channel coding. By virtue of our optimal joint source-channel 
coding criterion, the total distortion will be twice the distortion 
due to channel coding. Hence, given an end-to-end limit on 
the distortion, we can get the minimum packet length to be 
chosen from Fig. 3. The squared distortion metric with k = 4, 
R = 10 and packet erasure probability S = 10~ 3 was chosen. 
The o (r) and o (R) terms were neglected in the terms for 
distortion due to noisy channel decoding in J9j and (|22|) .The 
asymptotic nature of the curve indicates that large packet size 
is not required for packet erasure channels with small erasure 
probabilities. 
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upper (UB) and lower bounds (LB) on the optimal channel 
plotted for various values of packet size P. 
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transmission over more sophisticated channel models. 
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IV. Conclusion 

The results presented in this paper provide a mechanism for 
optimal concatenation of source and channel coders. Analytic 
results are provided for lower and upper bounds for a binary 
erasure channel and for packet erasure channels. The results 
on packet erasure channel enable us to obtain the bounds 
on the packet size for a specified bound on the distortion 
and given packet erasure probability. Alternately, for a given 
packet erasure probability, we can find bounds on the channel 
encoding rate for various packet lengths. By studying the 
optimal rate allocation for a bit and packet erasure channel, 
one can apply these results for transmission in a wide range 
of scenarios, including wireline channels with congestion. In 
future work, these bounds should be expanded to include 



